Skip to content

fix: relabel at-or-below-cap data at the capped pixel scale - #431

Merged
Jammy2211 merged 1 commit into
mainfrom
feature/small-datasets-loader-pixel-scales
Aug 3, 2026
Merged

Jammy2211 merged 1 commit into
mainfrom
feature/small-datasets-loader-pixel-scales

Conversation

@Jammy2211

Copy link
Copy Markdown
Collaborator

Summary

cap_array_2d_for_small_datasets handled one case and silently dropped the other. Data larger than the 16x16 PYAUTO_SMALL_DATASETS cap was cropped and rebuilt at SMALL_DATASETS_PIXEL_SCALES (0.6); data already at-or-below the cap early-returned, keeping the caller's uncapped pixel_scales (0.1). A capped simulator writes its data at 0.6, so the loader mislabelled the frame 6x — ±0.8" instead of ±4.8" for a 16x16 field.

Off-centre galaxies then fell outside the mislabelled frame, their non-negative linear intensity solve correctly returned exactly 0.0, total_luminosity became 0, and min(5 * 0.5 * 0**0.6, 5.0) collapsed a UniformPrior to lower == upper == 0.0. The PriorException surfaced four steps downstream in autolens_workspace/scripts/group/slam.py:321, naming neither the loader nor the pixel scale.

The at-or-below-cap branch now rebuilds the Array2D at the capped scale, mirroring the crop branch. Shape is preserved — that branch must never crop. Rebuilding is required rather than returning a corrected scalar: the Array2D is constructed by the caller before the call and carries its own geometry.

Fixes #430. Fixes the group/slam.py + group/slam.ipynb failures in PyAutoHeart Workspace Smoke run 30790463134.

API Changes

No signature changes. One behaviour change, confined to runs with PYAUTO_SMALL_DATASETS=1:
autoarray.util.dataset_util.cap_array_2d_for_small_datasets now returns SMALL_DATASETS_PIXEL_SCALES (0.6) and a rebuilt Array2D for input already at-or-below the cap, where it previously returned the caller's pixel_scales and the input array untouched. Shape is unchanged in that branch. Reached in practice through Imaging.from_fits, which routes data and noise_map (not psf) through it.

With the cap unset — every normal run — behaviour is byte-identical, and the crop branch is untouched.

See full details below.

Test Plan

  • pytest test_autoarray/ — 929 passed
  • scripts/group/slam.py (autolens_workspace) under the capped smoke profile, cleared dataset + output → exit 0, all six SLaM stages genuinely run (0 cached resumes; note output is namespaced under output/test_mode/)
  • Crop path unchanged — 209x209 cosmos_web_ring gives (16,16) @ 0.6 identically before and after
  • Normal uncapped operation unaffected — scripts/imaging/start_here.py (ENV: full_datasets) exit 0
  • Committed datasets audited: of 31 committed dataset FITS, exactly one is at-or-below the cap (slacs1430+4105/psf.fits, 11x11) and PSFs never route through the capper — so no committed dataset changes scale

Downstream effect on two parked scripts

  • imaging/features/scaling_relation/slam → now exit 0 (6 real searches). Its NEEDS_FIX park in autolens_workspace/config/build/no_run.yaml can be removed in a separate workspace PR.
  • multi_galaxy/features/scaling_relation/slam → gets past the 0.0-luminosity cause (real luminosities now measured) but then hits a separate latent script bug: slam.py:863 computes image_half_width from the script's own hardcoded pixel_scale (0.1) while the mask is built from dataset_full.pixel_scales (now correctly 0.6), producing an empty mask. Not a regression — on unpatched main it fails earlier with the documented 0.0-luminosity error. Stays parked; filed separately.
Full API Changes (for automation & release notes)

Changed Behaviour

  • autoarray.util.dataset_util.cap_array_2d_for_small_datasets(array_2d, pixel_scales) — when PYAUTO_SMALL_DATASETS=1 and array_2d.shape_native is at-or-below SMALL_DATASETS_SHAPE_NATIVE, now returns a rebuilt Array2D at SMALL_DATASETS_PIXEL_SCALES and that scale, instead of returning the inputs unchanged. Shape preserved; no cropping in this branch. Unchanged when the env var is unset, and unchanged for above-cap input.

Removed

  • None

Added

  • None

Migration

  • None required. Callers that already consume the returned pixel_scales (the documented contract, and what Imaging.from_fits does) need no change. Code that ignored the returned pixel_scales and reused its own literal was already relying on the buggy path under the cap; it should consume the returned value or read dataset.pixel_scales.

Generated by the PyAutoLabs agent workflow.

cap_array_2d_for_small_datasets handled one case and silently dropped the
other. Data larger than the 16x16 PYAUTO_SMALL_DATASETS cap was cropped AND
rebuilt at SMALL_DATASETS_PIXEL_SCALES (0.6); data already at-or-below the cap
early-returned, keeping the caller's uncapped pixel_scales (0.1). A capped
simulator writes its data at 0.6, so the loader mislabelled the frame 6x —
+/-0.8" instead of +/-4.8" for a 16x16 field.

Off-centre galaxies then fell outside the mislabelled frame, their non-negative
linear intensity solve correctly returned exactly 0.0, and the resulting
collapsed UniformPrior surfaced four steps downstream as a PriorException in
autolens_workspace scripts/group/slam.py, naming neither the loader nor the
pixel scale.

The at-or-below-cap branch now rebuilds the Array2D at the capped scale,
mirroring the crop branch. Shape is preserved — that branch must never crop.
Rebuilding is required rather than returning a corrected scalar: the Array2D
is constructed by the caller before the call and carries its own geometry.

Two unit tests asserted the early return as intended behaviour; both are
rewritten to assert the relabel-without-cropping. The env-unset test and both
crop-path tests are unchanged, guarding the scope of the fix.

Fixes #430

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@Jammy2211 Jammy2211 added the pending-release PR queued for the next release build label Aug 3, 2026
@Jammy2211
Jammy2211 merged commit 17885f3 into main Aug 3, 2026
2 checks passed
@Jammy2211
Jammy2211 deleted the feature/small-datasets-loader-pixel-scales branch August 3, 2026 18:03
@Jammy2211 Jammy2211 removed the pending-release PR queued for the next release build label Sep 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix: cap loader mislabels at-or-below-cap data pixel_scales

1 participant